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AGE  AND  PERFORMANCE  IN  AIR  TRAFFIC  CONTROL  SPECIALIST  TRAINING 


Allan  D.  VanDe venter  and  Nancy  E.  Baxter 


Introduction . 


Previous  research  on  the  relationship  between  age  at  entry  Into  the 
Federal  Aviation  Administration  (FAA)  air  traffic  control  specialist  (ATCS) 
occupation  and  performance  in  training  and  on  the  job  has  led  to  the 
conclusion  that  both  retention  rates  and  rated  job  performance  decrease 
dramatically  as  age  increases  (1,2).  This  research  became  the  basis  of  a 
1972  law  allowing  the  FAA  to  limit  entry  age  to  30  years  or  younger  for  en 
route  and  terminal  ATCS's.  Other  research  has  indicated  that  military 
instrument  flight  rules  (IFR)  air  traffic  control  (ATC)  experience  is 
predictive  of  higher  success  rates  in  FAA  ATCS  training  while  visual  flight 
rules  (VFR)  ATC  and  pilot  experience  are  not  (1,2).  Since  that  research  was 
conducted,  the  FAA  has  implemented  (i)  the  age  rule,  (ii)  selection  based  on 
aptitude  tests  for  all  entrants,  and  (ill)  a  rigorous  nonradar  control 
aptitude  screening  phase  that  must  be  passed  soon  after  entry  into  the  FAA. 
Research  based  on  the  new  program  has  confirmed  tie  findings  noted  above 
regarding  IFR  and  VFR  ATC  experience  (3). 

In  order  to  facilitate  the  agency's  recovery  from  the  loss  of  the 
majority  of  the  ATCS  work  force  following  the  strike  of  the  Professional  Air 
Traffic  Controllers  Organization  (PATCO),  exemptions  allowing  entry  up  to 
age  35  for  selected  groups  with  certain  kinds  of  ATC  or  pilot  experience 
were  considered  by  management  as  a  temporary  polioy  during  the  FAA  air 
traffic  recovery  program.  The  groups  identified  for  exemption  were  defined 
somewhat  differently  from  those  used  in  prior  research.  The  purposes  of  the 
present  study  were  (i)  to  examine  the  relationship  between  age  at  entry  and 
performance  in  the  current  initial  screening  phases  of  ATCS  training,  and 
(ii)  to  determine  whether  the  identified  ATC/pilot  groups  and  the 
no-experience  group  differed  in  relationships  between  age  and  performance  in 
those  phases. 

Method. 

Subjects .  A  biographical  questionnaire  was  administered  on  a  voluntary 
basis  at  the  beginning  of  the  initial  ATCS  screening  phases  to  trainees  who 
entered  the  FAA  Academy  between  June  1977  and  September  1982.  Performance, 
age,  and  preemployment  experience  data  were  available  on  8,573  trainees 
(88.5  percent  of  the  total  population).  Based  on  responses  to  questions 
regarding  pre-FAA  experience,  trainees  were  divided  into  the  following 
groups: 


wmj.  mmur*  muxm  wwwmlw  wj.? 

**•> 

v>*. 

& 


’rjV77srj^T*mrr3rrT£vr£v*zn « 


EXPERIENCE  GROUP  (EXP) 

IFR  -  Those  who  indicated  12  months  or  more  of  military  IFR  ATC  operations 
experience . 

VFR  -  Those  who  indicated  18  months  or  more  of  military  VFR  ATC  operations 
experience  and  who  did  not  meet  the  requirements  for  classification 
in  the  IFR  group. 

ATR  -  Those  who  indicated  they  had  an  airline  transport  rating  (ATR)  or  a 
commercial  pilot  rating  in  combination  with  an  IFR  rating  and  who  did 
not  meet  the  requirements  for  classification  in  the  IFR  or  VFR  groups. 

NO  EXPERIENCE  GROUP  (NO  EXP) 

NO  EXP  -  All  others  not  classified  above. 

Results . 


For  all  groups  combined,  the  results  indicated  a  dramatic  linear 
relationship  between  age  at  entry  and  pass/fail  status  at  the  end  of  initial 
ATCS  screening.  Those  22  years  old  and  younger  had  a  78-percent  pass  rate; 
those  31  years  old  or  older  had  a  52-percent  pass  rate.  Overall  pass  rates 
were  higher  for  those  in  the  EXP  group  than  for  those  in  the  NO  EXP  group, 
but  a  similar  decline  in  performance  with  age  was  evident  (Figure  1). 
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Figure  1.  Pass  rates  by  age  and  experience. 


TABLE  I.  NUMBER  ENTERING  (N)  AND  PASS  RATE  (P) 
BY  AGE  AND  EXPERIENCE  GROUPS 


AGE  GROUP  NO  EXP 
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The  pass  rates  by  age  and  experience  type  are  suamarized  in  Table  I. 

An  analysis  of  variance  of  experience  type  with  the  Academy  nonradar  score 
as  the  dependent  measure  was  significant  (p<.001).  Tukey  posthoc  tests 
indicated  significant  differences  be ween  the  NO  EXP  and  IFR/ATR  groups  and  a 
significant  difference  between  the  VFR  and  IFR  groups.  No  other  group 
differences  were  significant. 
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Figure  2.  Pass  rates  by  age  and  experience  type. 
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As  can  be  seen  in  Figure  2,  the  performance  of  the  VFR  experience  group 
is  variable  across  the  age  groups  (this  may  be  due  to  the  much  smaller 
sample).  The  IFR  and  ATR  groups  show  very  similar  performance.  All  groups 
show  a  similar  trend  with  age. 
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Figure  3*  Age  -  pass  rate  regression  lines. 


The  calculated  regression  lines  of  age  and  pass  rates  are  plotted  in 
Figure  3  for  the  combined  EXP  group  and  the  NO  EXP  group.  Inspection  of 
those  lines  indicates  that  the  performance  of  the  EXP  group  matches  that  of 
the  NO  EXP  group  approximately  4-5  years  younger  (e.g . ,  the  pass  rate  of  the 
32-year-old  EXP  group  matches  that  of  the  27-year-old  NO  EXP  group). 

Discussion . 


Performance  in  ATCS  nonradar  screening  shows  a  very  marked  inverse 
relationship  with  age  at  entry  into  the  FAA  ATCS  training  program.  This 
drop  is  even  more  dramatic  considering  the  fact  that  (i)  the  group  is 
relatively  young  (mean  age  26),  (ii)  the  group  has  been  screened  for 
aptitude  and  extreme  selection  has  taken  place,  and  (iii)  the  age  range  is 
attenuated.  The  cause (s)  of  such  a  strong  age-related  effect  is/are  not 
clear.  It  is  probable  that  they  are  cognitive  in  nature  and  have  an 
inhibiting  effect  on  early  performance  in  learning  the  ATCS  job. 


The  superior  performance  of  the  IFR  group  is  similar  to  previoua 
findings  (3)*  The  difference  between  the  VFR  and  NO  EXP  groups  is  larger  in 
this  study  (which  used  18+  months  experience  to  define  the  VFR  group)  than 
the  findings  of  previous  studies  (3)  that  used  a  6-month  definition.  Since 
the  present  study  found  no  significant  differences  in  scores  between  VFR  and 
NO  EXP,  and  since  the  VFR  pass  rates  were  highly  variable  by  age,  the 
present  findings  are  viewed  as  inconclusive  regarding  performance 
differences  between  VFR  and  NO  EXP  groups.  The  ATR  group  showed  similar 
Academy  performance  relative  to  the  NO  EXP  group  as  in  previous  research 
(1);  however,  it  should  be  noted  that  the  same  research  showed  a  higher 
poat-Academy  attrition  for  pilots;  overall,  they  had  a  retention  rate  equal 
to  that  of  the  nonexperienced  group. 

The  regression  lines  of  the  EXP  and  NO  EXP  groups  are  shifted  by 
approximately  5  years  (among  the  older  trainees).  This  finding  was  viewed 
as  lending  support  to  the  strike  recovery  program  change  in  ATCS  selection 
procedures  that  involved  a  temporary  adjustment  of  the  age  rule  to  35  years 
for  the  identified  experience  group  as  a  whole. 
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A  COMPARISON  OF  PRESTRIKE  AND  POSTSTRIKE  ATCS  TRAINEES:  BIOGRAPHIC 
FACTORS  ASSOCIATED  WITH  ACADEMY  TRAINING  SUCCESS 


William  E.  Collins,  Carol  A .  Manning, 
and  Deborah  K.  Taylor 


Introduction. 


The  October  1981  strike  of  the  members  of  the  Professional  Air  Traffic 
Controllers  Organization,  and  their  subsequent  firing  by  the  President,  led 
to  the  air  traffic  strike  recovery  program  which  included  the  unprecedented 
hiring  and  basic  training  of  over  8,000  ATCS  applicants  in  a  2-year  period. 
This  study  examined  the  relationship  between  autobiographic  factors  reported 
by  poststrike  air  traffic  control  specialist  (ATCS)  trainees  and  their 
success  in  the  FAA  Academy  screening  program  in  Oklahoma  City. 

One  purpose  of  this  study  was  to  compare  the  poststrike  trainees  with  a 
group  of  prestrike  trainees  to  determine  whether  they  differed  significantly 
on  any  biographic  variables.  It  was  anticipated  that  trainees  who  entered 
the  Academy  after  the  strike  would  differ  from  those  who  entered  prior  to 
the  strike.  Before  the  strike,  nearly  50  percent  of  the  trainees  were 
ex-military  air  traffic  controllers.  However,  after  the  strike,  it  was 
expected  that  the  group  of  Academy  trainees  would  be  much  less  experienced 
because  (i)  publicity  about  the  high  salaries  of  ATCS's  might  attract  a 
different  group  of  applicants,  (ii)  the  number  of  applicants  required  to 
replace  the  striking  controllers  far  exceeded  the  number  of  applicants  with 
prior  military  ATC  experience,  and  (iii)  applicants  with  any  substantive 
ATCS  experience  bypassed  the  Academy  screening  program  and  went  directly 
into  field  training  as  a  temporary  strike  recovery  strategy. 

The  second  purpose  of  this  study  was  to  identify  biographical  factors 
related  to  success  in  the  screening  program  for  possible  use  in  efforts  to 
improve  recruitment  of  potentially  successful  applicants.  In  a  previous 
study  (2),  prestrike  trainees'  responses  to  a  biographical  questionnaire 
(BQ)  were  used  to  identify  several  factors  which  were  significantly  related 
to  performance  in  the  screening  program.  Among  these  factors  were  age, 
prior  ATC  experience,  high  school  math  grades,  overall  high  school  grades, 
the  number  of  times  the  applicant  took  the  ATC  aptitude  test,  and 
self-ratings  of  future  performance.  If  the  population  of  ATCS  applicants 
has  changed  since  the  strike,  then  biographical  factors  related  to 
performance  in  the  screening  program  may  also  have  changed. 

Method . 


Subjects  for  this  study  were  8,159  students  who  entered  training  at  the 
FAA  Academy  between  September  1981  and  April  1983.  A  voluntary  BQ  was 
administered  to  all  consenting  students  on  their  arrival  at  the  FAA  Academy, 
along  with  a  battery  of  research  tests  designed  to  improve  ATCS  selection 
procedures.  The  BQ  contained  96  items  that  inquired  about  such  areas  as 
types  of  previous  experience,  education,  degree  of  interest  in  aviation, 
number  of  times  the  ATCS  aptitude  test  was  taken,  whether  the  student 


prepared  for  the  ATCS  aptitude  test,  reason  for  wanting  to  become  an  ATCS, 
likelihood  of  remaining  in  the  ATCS  occupation  for  some  time,  importance  of 
factors  influencing  career  choice,  whether  right  or  left  handed,  economic 
situation  of  family  while  growing  up,  marital  status,  and  tobacco  and 
alcohol  usage.  Data  regarding  the  trainee's  age,  sex,  minority  status,  and 
pass/fail  status  in  the  screening  program  were  also  available. 

Results . 

Comparison  of  Prestrike  and  Poststrike  Samples.  The  first  analysis 
compared  demographic  characteristics  of  the  poststrike  sample  described 
above  with  a  sample  of  6,059  prestrike  trainees  who  completed  another 
version  of  the  BQ.  These  statistics  are  shown  in  Table  I.  The  table  shows 
that,  while  the  trainees  in  both  groups  have  about  the  same  average  age, 
they  differ  on  other  characteristics.  A  slightly  lower  percentage  of 
females  entered  the  FAA  Academy  poststrike.  The  percentage  of  minority 
entrants  dropped  6  percent  to  about  half  of  their  previous  level  of 
representation.  The  percentage  of  entrants  having  college  degrees  increased 
about  10  percent  since  the  strike.  The  largest  differences  between  the 
samples  can  be  seen  in  the  percentage  of  trainees  who  had  prior  ATC 
experience  and  the  percentage  receiving  veteran  preference  points.  While 
before  the  strike,  only  about  30  percent  of  the  trainees  had  no  prior 
aviation  experience,  since  the  strike,  over  two-thirds  of  the  trainees  had 
no  aviation  experience.  Before  the  strike,  about  60  percent  of  the  trainees 
received  veteran  preference  points,  but  since  the  strike,  that  percentage 
decreased  to  just  over  30  percent. 

TABLE  I.  COMPARISON  OF  ACADEMY  PRESTRIKE  AND  POSTSTRIKE  ATCS  TRAINEES 


PRESTRIKE 

POSTSTRIKE 

(N=6,059) 

(N=8, 159) 

AVERAGE  AGE 

26.5 

26.3 

SEX 

83 

85 

t  MALES 

17 

15 

%  FEMALES 

EDUCATION 

14 

9 

%  HIGH  SCHOOL  ONLY 

48 

44 

%  SOME  COLLEGE 

38 

48 

%  COLLEGE  DEGREE 

MINORITY 

13 

7 

%  MINORITIES 

REPRESENTATION 

87 

93 

%  NONMINORITIES 

PRIOR 

30 

68 

%  NO  PRIOR  EXPERIENCE 

EXPERIENCE 

20 

17 

t  AVIATION  EXPER  ONLY 

50 

15 

%  PRIOR  ATC  EXPERIENCE 

VETERAN  STATUS 

60 

31 

%  RECEIVING  VET  PREF  POINTS 

40 

69 

%  NOT  RECEIVING  VET  POINTS 

Relating  Biographical  Factors  to  Performance. 

Restriction  of  Sample.  The  purpose  of  this  part  of  the  study  was  to 
examine  the  relationship  between  responses  to  BQ  items  and  success  in  the 
Academy  screening  program,  with  the  goal  of  improving  recruitment 
procedures.  The  analyses  for  this  part  of  the  study  used  data  obtained  from 
only  the  poststrike  trainees.  Certain  trainees  were  eliminated  from  the 
analyses  because  they  were  not  representative  of  typical  poststrike  ATCS 
applicants.  For  example,  trainees  who  entered  the  Academy  between  September 
1981  and  December  1981  were  eliminated  because  the  confusion  surrounding  the 
strike  and  the  Academy's  need  to  accommodate  a  large  number  of  trainees 
suddenly  may  have  affected  trainees'  performance  (and  responses  to  the  BQ) 
in  an  unusual  way.  Additionally,  trainees  who  claimed  to  have  prior  ATC 
experience  were  eliminated  because  recruitment  efforts  will  likely  be 
directed  towards  attracting  applicants  who  have  no  previous  experience. 
Trainees  who  repeated  the  screening  program  were  also  eliminated  because 
their  performance  is  usually  atypical  of  that  of  trainees  with  no  prior 
entry.  Finally,  participants  in  special  entry  programs  involving 
pretraining  in  ATC  were  eliminated  from  the  analyses  because  their 
performance  is  also  atypical  of  that  of  other  trainees.  The  data  from  the 
BQ  were  then  recoded  so  that  all  responses  were  either  ordinal  or 
dichotomous . 

After  eliminating  nonrepresentative  trainees  from  further  analysis, 
those  remaining  represented  the  kind  of  inexperienced  applicants  toward  whom 
a  recruitment  program  would  likely  be  targeted.  Thus,  if  a  relationship 
were  found  between  biographical  factors  and  success  in  the  screening 
program,  the  information  might  be  a  useful  supplement  for  ourrent  recruiting 
practices.  Furthermore,  the  FAA  has  a  special  interest  in  recruiting 
qualified  minority  applicants.  If  biographical  factors  different  from  those 
which  predicted  Academy  success  for  nonminority  trainees  were  found  to 
predict  Academy  success  for  minority  trainees,  that  information  could  be 
used  to  identify  target  areas  where  recruitment  should  be  emphasized,  or  to 
identify  minorities  who  should  be  encouraged  to  submit  an  application. 

Factors  Related  to  Pass/Fail  Status .  Rather  than  include  all  96  BQ 
items  in  a  regression  model,  it  was  decided  to  identify  a  subset  of  items 
significantly  related  to  the  trainees'  pass/fail  status  and  use  these  items 
in  subsequent  regression  analyses.  Identification  of  related  items  was 
based  on  chi-square  and  correlational  analyses  of  the  relationship  between 
each  variable  and  pass/fail  status.  A  .005  level  of  significance  was 
adopted  for  analyses  for  the  nonminority  group  because  the  large  number  of 
observations  for  the  group  (4,673)  allows  detection  of  very  small 
differences.  The  minority  group  contained  212  trainees}  consequently,  a 
.05  significance  level  was  adopted  for  those  analyses. 

Table  II  shows  the  items  related  to  pass/fail  status  for  both 
minorities  and  nonminorities.  Several  items  were  removed  from  consideration 
because  they  related  to  factors  not  easily  discussed  during  a  recruitment 
interview.  Among  these  items  were  how  long  the  candidates  had  been  married, 
whether  or  not  they  smoked,  and  whether  they  were  right  or  left  handed. 


TABLE  II.  FACTORS  POSITIVELY  RELATED  TO  PASS/FAIL  STATUS 
FOR  MINORITY  AND  NONMINORITY  TRAINEES 


MINORITY  TRAINEES  (N=212) 

+  Had  good  high  school  grades  in  (1)  physical  science,  (2)  biology, 

(3)  English,  (4)  social  studies,  and  (5)  business  courses 
Had  high  overall  high  school  grade  average 
Attended  college  (graduating  from  college  is  not  related) 

*  Younger  age 

+  Did  not  prepare  for  the  ATCS  aptitude  test 

+  Predicted  own  performance  would  be  at  the  top  of  all  ATCS's  in  country 
+  Often  flies  in  an  airplane 

+  Success  at  the  Academy  was  also  marginally  related  to: 

Not  being  in  the  Armed  Forces 
Good  high  school  math  grades 

NONMINORITY  TRAINEES  (N=4,673) 

+  Obtained  high  school  diploma  granted  by  school 

Had  good  high  school  grades  in  (1)  math,  (2)  physical  science, 

(3)  biology,  (4)  social  studies,  (5)  English,  (6)  business,  (7) 
physical  education,  and  (8)  vocational  or  agricultural  courses. 

Had  high  overall  high  school  grade  average 

Graduated  from  college  (attending  college  is  not  related) 

+  Younger  age 

+  Was  not  a  member  of  the  Armed  Forces 

+  Took  the  ATCS  aptitude  test  only  once 

+  Rated  chances  of  staying  in  ATC  work  <  3  years  unlikely 

♦  Predicted  performance  would  be  at  the  top  of  all  ATCS's  in  country 
+  Assigned  low  importance  ratings  to  following  factors: 

Work  which  benefits  others 

Work  in  which  you  can  create  something  new 

Work  you  can  do  as  fast  or  slow  as  you  want 

Work  in  which  you  are  sure  of  having  a  Job  even  in  hard  times 

Ability  to  control  the  amount  of  work  you  have  to  do. 

+  Assigned  a  low  importance  weight  to  the  factor 

"Work  in  which  you  get  a  good  salary"  (marginal  relationship) 


It  can  be  seen  that  high  school  grades  and  overall  high  school  grade 
averages  predict  success  for  both  minority  and  nonminority  trainees. 

However,  while  grades  in  math  were  most  highly  correlated  with  Aoademy 
success  for  nonminority  trainees,  they  were  the  least  predictive  of  high 
school  grades  for  minorities.  The  high  school  oourse  for  whioh  the  grades 
best  predicted  minority  success  was  physical  science.  Graduating  from 
college  was  related  to  success  for  nonminority  trainees,  but  not  for 
minorities.  However,  attendance  in  college  was  related  to  minority  suooess. 


Several  studies  have  shown  that  age  at  entry  has  a  high  negative 
correlation  with  success  in  the  screening  program  (1),  and  that  relationship 
was  found  to  occur  with  these  samples.  Trainees  who  had  not  been  in  the 
Armed  Forces  were  more  successful  than  those  who  had  been  in  the  military. 
Also,  in  both  groups,  trainees  who  predicted  they  would  rate  among  the  top 
group  of  controllers  throughout  the  country  performed  better  than  those  who 
predicted  they  would  be  average  controllers. 

Some  factors  were  related  to  Academy  success  for  only  the  nonminority 
trainees.  Nonminority  trainees  who  considered  salary  and  job  stability 
unimportant  or  were  not  concerned  about  their  ability  to  control  their  work 
had  higher  pass  rates  than  those  who  were  concerned  about  these  factors. 

The  variables  involving  taking  the  OPM  aptitude  test  more  than  one  time 
and  having  prepared  for  the  test  are  negatively  related  to  pass/fail  status 
because  previous  exposure  to  the  test  improves  subsequent  test  performance 
but  does  not  improve  the  likelihood  of  success  in  the  screening  program. 
Analyses  of  testing  data  have  shown  that  (i)  as  an  applicant  repeats  the 
test,  his  or  her  test  performance  usually  improves,  and  (ii)  performance  at 
the  Academy  is  more  closely  related  to  the  original  than  to  the  final  OPM 
test  score.  Thus,  nonminority  candidates  who  repeat  the  test  to  earn  a 
higher  score,  often  do  not  pass  the  screen.  On  the  other  hand,  many 
minority  applicants  do  not  repeat  the  test  but  are  coached  before  their 
first  attempt  by  someone  who  has  access  to  it.  Coaching  may  improve  test 
performance  but  does  not  improve  the  likelihood  of  successfully  completing 
the  screening  program. 

Regression  Analyses . 

The  variables  identified  above  for  each  group  were  entered  as 
predictors  into  separate  regression  analyses  for  the  minority  and 
nonminority  trainees.  A  series  of  model  comparisons  was  then  performed  to 
identify  a  set  of  variables  that  (i)  would  predict  an  Academy  pass/fail 
status  as  well  as  a  full  model  containing  a  greater  number  of  variables,  and 
(ii)  consists  of  information  that  could  be  easily  obtained  by  a  recruiter 
during  an  interview. 

Minorities .  For  the  minority  trainees,  a  "full"  regression  model 
containing  all  the  variables  identified  in  Table  I  had  a  squared  multiple 
correlation  of  .22,  which  was  significantly  different  from  zero, 

F( 16, 183)=3>23,  P  <  .0001.  A  "reduced"  regression  model  containing  only  the 
variables  age,  physical  science  high  school  grades,  and  self-rating  of 
performance  potential  (which  had  a  squared  multiple  correlation  of  .13)  was 
then  compared  with  the  full  model;  the  full  model  predicted  pass/fail 
status  no  better  than  the  reduced  model,  F(13,l83)  =  1.69,  p  >  .05. 


These  results  suggest  that  using  information  about  the  trainees'  age, 
high  school  physical  science  grades,  and  self-rating  of  future  success  would 
be  just  as  prediotive  of  Academy  success  as  the  "full"  set  of  variables  and 
is  more  parsimonious  in  the  reoruitment  situation.  Table  III  shows  the  set 
of  variables  identified  in  the  regression  analysis  and  their  associated  beta 
weights.  The  beta  weights  show  that  age  is  the  most  important  variable  and 


is  negatively  related  to  pass/fail  status.  Physical  science  grades  were 
almost  as  important  as  age  in  predicting  Academy  success,  while  the 
self-rating  was  somewhat  less  important  than  the  other  two  variables. 

TABLE  III.  SET  OF  VARIABLES  SIGNIFICANTLY  PREDICTING  ACADEMY 
PASS/FAIL  STATUS  FOR  MINORITY  CANDIDATES 

Variable  Beta  weight 


Age  -.22 

Physical  science  high  school  grades  .20 

Of  all  air  traffic  controllers,  where  .15 

do  you  think  you  will  perform? 


Nonminorities .  For  nonminority  candidates,  the  "full"  regression  model 
containing  all  the  variables  identified  in  Table  II  had  a  squared  multiple 
correlation  of  .09,  which  was  significantly  different  from  zero,  F( 23, 4623) 

=  21.01,  p  <  .0001.  The  full  model  was  compared  with  a  "reduced"  model 
containing  only  the  variables  age,  math  high  school  grades,  and 
self-assessment  of  future  performance.  This  reduced  regression  model  had  a 
squared  multiple  correlation  of  .07,  a  reduction  of  only  2  percent  from  the 
proportion  of  variance  accounted  for  by  the  full  model.  Table  IV  shows  the 
variables  contained  in  the  reduced  model  and  their  beta  weights.  It  is 
apparent  that  the  high  school  math  grade  is  the  most  important  variable 
included  in  the  model,  while  age  is  second,  and  the  self-assessment  is 
third.  These  results  suggest  that  a  candidate  who  had  good  high  school  math 
grades,  is  young,  and  is  confident  about  his  or  her  chances  of  success  as  an 
ATCS  is  likely  to  succeed  in  the  FAA  Academy  screening  program. 

TABLE  IV.  SET  OF  VARIABLES  SIGNIFICANTLY  PREDICTING  ACADEMY 
PASS/FAIL  STATUS  FOR  NONMINORITY  CANDIDATES 
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Conolualona . 

Since  the  strike,  the  composition  of  the  population  applying  for 
positions  as  ATCS's  has  changed.  More  than  two- thirds  of  the  poststrike 
applicants  have  no  prior  experience  in  aviation.  Consequently,  recruiting 
efforts  may  have  to  be  directed  towards  attracting  qualified  candidates  from 
areas  outside  of  aviation. 

The  variables  found  to  be  predictors  of  pass /fall  status  in  the  Academy 
screening  program  could  be  incorporated  in  recruitment  efforts.  Recruiters 
might  target  their  efforts  to  attract  qualified  candidates  towards  colleges 
or  junior  colleges  because  such  institutions  are  likely  to  have  a  good 
supply  of  young  applicants  who  earned  good  grades  in  high  school. 

Recruiters  should  attempt  to  encourage  candidates  to  apply  while  they  are 
young  because  the  likelihood  of  success  in  the  screening  program  decreases 
as  they  grow  older.  They  should  refrain  from  training  applicants  to  take 
the  ATCS  aptitude  test  before  their  official  testing  session  and  should 
refrain  from  encouraging  applicants  to  repeat  the  test  if  they  fail  the 
first  time. 

This  study  has  identified  a  set  of  variables  that  could  be  used  by 
recruiters  to  identify  both  minority  and  nonminority  candidates  who  might  be 
successful  in  the  FAA  Academy  screening  program.  Age,  high  school  grades, 
and  self-assessment  of  future  performance  were  the  most  important  predictors 
of  Academy  success  for  both  minorities  and  nonminorities,  but  the  order  of 
importance  depended  on  which  group  was  considered.  The  most  important 
predictor  of  minority  pass/fail  status  was  age,  with  physical  science  high 
school  grades  second.  For  nonminorities,  the  most  important  predictor  of 
pass/fail  status  was  the  math  high  school  grade,  and  age  was  second  in 
importance . 

This  information  may  provide  a  guideline  to  aid  in  identifying  various 
categories  of  applicants  who  might  be  successful  in  the  FAA  Academy 
screening  program.  The  variables  described  in  this  study  were  not  intended 
to  be  used  for  screening  purposes,  but  instead  to  target  individuals  who 
might  be  most  strongly  encouraged  to  proceed  with  the  application  process. 
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A  FOLLOWUP  EVALUATION  OF  THE  NEW  APTITUDE  TESTING  PROCEDURES  FOR 
SELECTION  OF  FAA  AIR  TRAFFIC  CONTROL  SPECIALISTS 

Allan  D.  VanDeventer,  Ph.D 


Introduction. 

The  air  traffic  control  specialist  (ATCS)  developmental  process  in  the 
en  route  and  terminal  options  may  be  viewed  as  consisting  of  four  parts, 
each  of  which  must  be  successfully  completed  in  order  to  achieve  journeyman 
status  as  a  radar  ATCS  (see  Table  I).  The  first  part  consists  of  two  stages 
of  selection  screening,  one  prior  to  hire  and  the  other  immediately 
following  entry  into  the  Federal  Aviation  Administration  (FAA).  The 
rationale  for  these  screening  procedures  is  to  determine  as  early  as 
possible  whether  or  not  the  person  has  sufficient  aptitude  to  complete  the 
long  and  costly  on-the-job  training  (OJT)  necessary  to  become  a  journeyman 
controller . 

TABLE  I.  THE  ATCS  DEVELOPMENTAL  PROCESS 
PART  LOCATION  EVALUATION  METHODS 


1.  SELECTION  SCREENING 

FIRST  STAGE 
SECOND  STAGE 

2.  NONRADAR  TRAINING 

3.  PLACEMENT  SCREENING 

4.  RADAR  TRAINING 


OFFICE  OF  PERSONNEL  MGMT.  APTITUDE  TESTS 
FAA  ACADEMY  NONRADAR  JOB  SIMULATION 


FIELD  FACILITY 
FAA  ACADEMY  RADAR 
FIELD  FACILITY 


OJT  AND  CHECKOUT 
JOB  SIMULATION 
OJT  AND  CHECKOUT 


In  October  1981,  the  Office  of  Personnel  Management  (OPM),  in 
conjunction  with  the  FAA,  implemented  a  new  first-stage  selection  procedure 
for  ATCS's.  As  was  the  case  for  the  prior  OPM  selection  procedure 
(implemented  in  1964 — reference  1),  the  new  procedure  is  based  primarily  on 
aptitude  tests  that  have  been  demonstrated  to  have  validity  for  prediction 
of  successful  performance  in  the  second  stage  of  screening— the  ATCS 
nonradar  screening  phases  at  the  FAA  Academy  (2,3).  The  new  procedure 
retained  the  Abstract  Reasoning  Test  (ABSR),  OPM  Test  157,  that  had  been 
used  in  the  prior  procedure.  A  new,  highly  job-related  aptitude  test,  the 
Multiplex  Controller  Aptitude  Test  (MCAT),  OPM  Test  510,  was  added  in  two 
forms  (MCATA  and  MCATB)  as  was  a  new  procedure  for  granting  credit  for  prior 
ATCS  experience,  the  Occupational  Knowledge  Test  (OKT),  OPM  Test  512  (4). 

In  the  new  procedure,  an  applicant's  total  weighted  score  (TWS)  is 
computed  by  the  formula: 
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TVS  =  2»(MCATA)  +  2«(MCATB)  +  1»(ABSR) 


which  was  derived  based  on  the  research  preliminary  to  implementation  of  the 
new  battery.  This  is  then  converted  to  a  scale  with  a  mean  of  70  and  a 
maximum  of  100— -the  transmuted  composite  (TMC) — by  the  formula: 

TMC  =  70  ♦  (TVS- 1 60)  •  (30/95) 

where  160  is  the  mean  TWS,  30  is  the  range  between  the  TMC  mean  (70)  and  the 
TMC  maximum  score  (100),  and  95  is  the  range  between  the  mean  TWS  (160)  and 
the  maximum  TWS  (255).  The  TWS  mean  and  maximum  were  those  used  in  the 
research  on  the  battery  prior  to  implementation.  The  mean  of  70  on  the  TMC 
is  the  cutoff  designated  by  0PM  for  the  part  of  qualification  based  on 
aptitude  testing.  This  is  intended  to  qualify  approximately  50  percent  of 
those  tested  on  the  basis  of  aptitude  tests. 

For  those  achieving  a  TMC  of  70  or  greater,  extra  credit  points  are 
added  for  experience  based  upon  the  applicant's  performance  on  the  0KT  as 
follows : 


0KT  SCORE  POINTS 


0-51  0 

52-55  3 

56-59  5 

60  -  63  10 

64  +  15 


However,  the  result  may  not  exceed  100.  This  is  referred  to  as  the  earned 
rating.  To  this  number  0,  5,  or  10  veteran  preference  points  are  added  to 
arrive  at  the  selection  rating  used  to  rank  and  select  candidates  (final 
rating).  Veteran  preference  points  are  not  predictive  of  performance  on  the 
job,  rather  they  are  added  as  a  matter  of  national  policy /law.  This  final 
figure  and  other  basic  qualifications  (e.g.,  experience/education,  medical) 
are  used  to  establish  eligible  applicants  on  registers  for  selection. 


The  purpose  of  this  study  was  to  evaluate  the  results  of  the  initial 
operational  administration  of  the  new  selection  battery  relative  to  the 
previous  selection  procedure.  The  performance  in  initial  (pass/fail) 
nonradar  qualification  training  at  the  FAA  Academy  was  used  as  the 
criterion. 


Methods. 

Subjects  were  3,746  ATCS  trainees  seleoted  from  the  new  procedure  and 
1,780  (2  did  not  have  full  data  sets)  ATCS  trainees  selected  from  the  prior 
procedure  during  the  period  August  1981  through  December  1982.  Entrants 
during  this  time  period  were  selected  since  (i)  during  the  initial 
two-thirds  of  this  time  period,  the  transition  from  the  old  procedure  to  the 
new  procedure  was  in  process  and,  thus,  selections  were  occurring  from  both 
procedures,  (ii)  enough  selections  on  the  new  procedure  needed  to  occur  to 
get  a  reasonable  sample  across  the  new  rating  distribution,  and  (iii)  after 


December  1982,  the  criterion  (Academy  nonradar  phase  scores)  evidenced  some 
change  due  to  grading  standardization  improvements  at  the  Academy.  Numbers 
of  subjects  included  in  various  analyses  varied  slightly  due  to  occasional 
missing  data.  (NOTE:  All  percentages  are  rounded  to  whole  numbers.) 

Results. 

Data  were  available  to  compare  the  earned  rating,  veteran  preference 
points,  and  final  ratings  for  the  old  and  new  selection  procedures.  For 
Academy  entrants,  the  means,  standard  deviations,  and  restricted  validity 
coefficients  (using  both  Academy  score  and  pass/loss  status  as  criteria)  for 
the  two  procedures  are  detailed  in  Table  II.  The  earned  rating  shows 
substantially  higher  correlations  for  the  new  selection  procedure,  while 
final  rating  correlations  are  essentially  the  same.  This  effect  is  clearly 
due  to  the  strong  negative  correlation  for  veteran  preference  points  in  the 
new  selection  procedure  sample. 

TABLE  II.  OLD  AND  NEW  SELECTION  PROCEDURE  PARAMETERS 
AND  VALIDITY  COEFFICIENTS 


EARNED 

VETERAN 

FINAL 

RATING 

POINTS 

RATING 

OLD  SELECTION  PROCEDURE 

N  =  1,778  (INCLUDES  WITHDRAWALS) 
RESTRICTED  MEAN 

89.39 

1.70 

91.09 

RESTRICTED  STANDARD  DEVIATION 
RESTRICTED  CORRELATION 

6.46 

2.50 

6.78 

WITH  PASS/LOSS  STATUS 

.26 

.00 

.24 

N  =  1,541  (EXCLUDES  WITHDRAWALS) 
RESTRICTED  MEAN 

89.68 

1.75 

91.42 

RESTRICTED  STANDARD  DEVIATION 
RESTRICTED  CORRELATION 

6.44 

2.50 

6.77 

WITH  ACADEMY  SCORE 

.32 

-.03 

.30 

NEW  SELECTION  PROCEDURE 

N  a  3,746  (INCLUDES  WITHDRAWALS) 
RESTRICTED  MEAN 

88.18 

1.58 

89.76 

RESTRICTED  STANDARD  DEVIATION 
RESTRICTED  CORRELATION 

5.23 

2.59 

5.19 

WITH  PASS/LOSS  STATUS 

.31 

-.15 

.24 

N  a  3,417  (EXCLUDES  WITHDRAWALS) 
RESTRICTED  MEAN 

88.38 

1.54 

89.92 

RESTRICTED  STANDARD  DEVIATION 
RESTRICTED  CORRELATION 

5.21 

2.56 

5.20 

WITH  ACADEMY  SCORE 

.40 

-.18 

.31 

The  differences  due  to  veteran  preference  points  are  likely  due  to 
changes  in  the  population  of  persons  entering  the  Academy  program  during 
this  period  of  time.  As  part  of  the  FAA  recovery  from  the  Professional  Air 
Traffic  Controllers  Organization  strike,  those  with  2  years  or  more  of 
pre-FAA  ATC  experience  were  hired  directly  into  FAA  ATC  field  facilities, 
bypassing  the  Academy  screening  programs.  Previous  research  has  shown  that 
those  with  such  background  experience  are  more  successful  at  the  Academy 
(5).  It  is  also  likely  that  this  exemption  affected  the  new  selection 
procedure  more  than  it  did  the  old  selection  procedure,  since  it  was 
implemented  during  the  switchover.  Additional  analyses  were  conducted  to 
determine  if  this  was  the  case  (see  Table  III). 

TABLE  III.  PASS  RATES  BY  VETERAN  PREFERENCE  AND  EXPERIENCE 


SELECTION  PROCEDURE 


OLD 

NEW 

N  PASS  * 

N  PASS  * 

NO  VETERAN  PREFERENCE 

1,194  (67%) 

5b* 

2,b58  (71*) 

67* 

VETERAN  PREFEERENCE 

584  ( 33%) 

5b* 

1,088  (29*) 

52* 

VETERAN  PREFERENCE 

WITH  ATC  EXPERIENCE 

293  (50*) 

62* 

263  (24*) 

65* 

WITH  PILOT  EXPERIENCE 

110  (19*) 

54* 

212  (19*) 

59* 

WITH  NO  EXPERIENCE 

181  (31*) 

47* 

613  (56*) 

44* 

Pass  rates  of  those  with  and  without  veteran  preference  points  were  the 
same  under  the  old  procedure,  while  under  the  new  procedure  nonveterans  had 
a  pass  rate  15  percentage  points  higher  than  veterans.  This  difference 
appears  to  be  due  to  differences  in  the  proportion  of  the  veterans  with  ATC 
experience.  More  than  twice  the  percentage  of  those  with  veteran  preference 
points  had  ATC  experience  under  the  old  procedure  (50  percent)  as  the  new 
procedure  (24  percent).  Moreover,  the  quality  of  the  ATC  experience  appears 
to  have  been  different.  Of  those  with  veteran  preference  and  ATC  experience 
under  the  old  procedure,  49  percent  had  more  than  6  months'  experience  in  an 
IFR  ATC  facility,  and  the  group  as  a  whole  had  pass  rates  6  percentage 
points  higher  than  those  for  nonveterans.  Under  the  new  procedure,  only  38 
percent  had  more  than  b  months'  experience  in  an  IFR  ATC  facility,  and  the 
group  pass  rate  was  2  percentage  points  lower  than  that  for  nonveterans. 

Unrestricted  population  estimates  of  the  standard  deviation  were  not 
available  for  the  old  selection  procedure  ratings,  so  it  was  not  possible  to 
correct  them  for  restriction  in  range  and  make  a  comparison.  However,  all 
scores  were  available  on  the  1981  applicant  population  for  the  new  selection 
procedure.  Unrestricted  standard  deviations  were  calculated,  and  the 
validity  coeffients  for  the  new  selection  procedure  were  corrected  for 
restriction  in  range  using  the  Thorndike  Two  Variable  Case  correction 
procedure  (b).  The  results  (see  Table  IV)  yielded  validity  coefficients  of 


|  TABLE  IV.  CORRECTED  CORRELATIONS  POR  NEW  SELECTION  PROCEDURE 


TMC 

EARNED 

RATING 

FINAL 

RATING 

APPLICANT  POPULATION  1981 

N  =  *4,9*9 

UNRESTRICTED  MEAN 

71.51 

71.82 

73.13 

UNRESTRICTED  STANDARD  DEVIATION 

11.8* 

12.20 

12.50 

8/81  -  12/82  ACADEMY  ENTRANTS 

N  =  3,7*6  (INCLUDES  WITHDRAWALS) 
RESTRICTED  CORRELATION 

WITH  PASS/LOSS  STATUS 

.26 

•  31 

.2* 

CORRECTED  CORRELATION 

WITH  PASS/LOSS  STATUS 

.5* 

.61 

.51 

N  s  3, *17  (EXCLUDES  WITHDRAWALS) 
RESTRICTED  CORRELATION 

WITH  ACADEMY  SCORE 

.33 

.*0 

.31 

CORRECTED  CORRELATION 

WITH  ACADEMY  SCORE 

.65 

.71 

.62 

0.51  (pass/loas  criterion)  and  0.62  (Aoadeay  score  criterion)  for  the  final 
rating.  Earned  rating  validity  ooeffiolenta  were  higher  (0.61  and  0.71)  due 
to  the  veteran  preference  points  effect.  TMC  validity  coefficients  were 
lower  than  earned  rating  coefficients.  This  pattern  indicates  that  the 
addition  of  extra  points  for  OKT  performance  increases  the  validity  of  the 
procedure . 


TABLE  V.  ACADEMY  PASS  RATES  BY  FINAL  RATING  RANGE 
FOR  NEW  AND  OLD  SELECTION  PROCEDURES 


SELECTION  PROCEDURE 


OLD  NEW 


FINAL 

RATING 

N 

PASS  * 

N 

PASS  * 

70-79 

56 

30* 

3* 

29* 

80-8* 

251 

38* 

587 

*** 

85-89 

*82 

*9* 

1  ,*99 

59* 

90-9* 

*62 

59* 

1,0*2 

70* 

95-99 

298 

6** 

*13 

78* 

1004- 

231 

75* 

171 

83* 

TOTAL 


1,780  56* 


3,7*6  63* 


20-  Legend 
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A  NEW 

0-| - r—  i - t- - 1 - 1 
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0PM  RATING 

Figure  1.  ATCS  pass  rates  by  selection  ratings  for 
poststrike  entrants  8/11/81  -  12/03/82 

Table  V  and  Figure  I  compare  the  pass  rates  by  rating  ranges  for  the 
new  and  old  selection  procedures.  Academy  pass  rates  for  the  new  procedure 
were  higher  for  all  but  the  70-79  rating  ranges.  The  overall  pass  rate  for 
those  selected  from  the  new  procedure  was  63  percent,  while  that  for  those 
selected  from  the  old  procedure  was  56  percent— an  increase  in  passes  of  7 
percent  for  this  period  of  time.  An  even  larger  increase  in  pass  rates 
would  be  possible  if  a  higher  cutoff  were  used  to  determine  eligibility. 

Conclusions . 

The  use  of  Job-related  aptitude  tests  such  as  the  MCAT  and  job 
knowledge  tests  such  as  the  0KT  have  increased  the  pass  rates  in  initial 
qualification  training,  thus  reducing  costs  due  to  attrition.  It  also 
appears  that  the  new  selection  procedure  has  resulted  in  improved  validity 
for  the  selection  of  ATCS  applicants,  although  the  increase  in  validity  was 
vitiated  by  the  reduced  level  of  pre-FAA  ATC  experience  among  veterans  in 
the  sample  studied.  Use  of  similar  aptitude  tests  and  procedures  for  the 
selection  of  ATCS's  in  the  military  (where  veteran  preference  points  are  not 
necessary)  and  in  other  countries  may  be  beneficial. 
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